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A piloted simulation was conducted to study handling qualities for capsule spacecraft 
entering the Earth’s atmosphere. Eight evaluation pilots, including six pilot astronauts, 
provided Cooper-Harper ratings, workload ratings, and qualitative comments. The 
simulation began after descending through the atmospheric entry interface point and 
continued until the drogue parachutes deployed. There were two categories of piloting tasks, 
both of which required bank angle control. In one task category, the pilot followed a closed- 
loop bank angle command computed by the backup guidance system to manage g-loads 
during entry. In the other task category, the pilot used intuitive rules to determine the 
desired bank angle independently, based on an open-loop schedule of vertical speed, Mach, 
and total energy specified at several range-to-target gates along the entry trajectory. Pilots 
were able to accurately track the bank angle guidance commands and steered the capsule 
toward the recovery site with essentially the same range error as the benchmark autopilot 
trajectory albeit with substantially higher propellant usage, and the handling qualities for 
this task were satisfactory. Another key result was that the complex piloting task of 
atmospheric entry could be performed satisfactorily, even in the presence of large 
dispersions, by controlling bank angle to follow a simple open-loop schedule. 


I. Introduction 

H andling qualities are those characteristics of a flight vehicle that govern the ease and precision with which a 
pilot is able to perform a flying task . 1 They are a manifestation of the interaction between various factors that 
influence pilot perception of how well a vehicle can be flown to accomplish a desired mission. These factors 
include the stability and control characteristics of the vehicle, the control systems that modify these characteristics, 
the inceptors (e.g., control stick or throttle lever) used by the pilot to transmit control commands, the visual cues 
from cockpit windows and display s/instrumentation that provide flight information to the pilot, and other cues (e.g., 
aural, tactile) that influence the pilot in the execution of the flying task. If a vehicle has good handling qualities, the 
pilot does not need to expend much effort compensating for vehicle deficiencies and can therefore reserve 
substantial cognitive capacity to successfully tackle anomalies or emergencies encountered in flight operations. 

The effects of the above factors on handling qualities have been studied in atmospheric flight vehicles over many 
decades . 2-6 Reference standards for the handling qualities of both fixed-wing aircraft 7 and rotary-wing aircraft 8 have 
been developed and are now in common use. Broadly speaking, these standards define a subset of the dynamics and 
control design space that predicts good handling qualities for a given vehicle type and flying task. For example, the 
standards may specify a range of combinations of damping and natural frequency for a large aircraft during landing 
that corresponds to satisfactory, acceptable, and unacceptable handling qualities. They provide design engineers 
with guidelines to optimize the flight control system and minimize potential human-vehicle interaction issues. 

At this time, no reference standards exist for spacecraft handling qualities. However, a modest body of work 
exists on handling qualities of piloted spacecraft. Most of this literature dates back to NASA’s Apollo program, 
covering studies conducted on lunar landing and orbital rendezvous/docking; there are also some publications on 
capsule entry . 9-12 The ability of pilots to successfully carry out missions with new generations of spacecraft will be 
determined in part by their handling qualities. Some flight operations may be fully automated, while others may be 
executed with a human pilot engaged in various levels of supervisory control including manual flying tasks . 14 
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Current NASA procedures require that human-rated spacecraft provide the capability for the crew to manually 
control the flight path and attitude with satisfactory handling qualities. 15 These procedures require that, even for 
flight operations that are nominally executed in a highly automated control mode, the control architecture must 
provide the capability for a human pilot to switch to a manual control mode - whether due to failure of an automated 
system or of some component of the spacecraft. In these cases of emergency reversion to manual control, where the 
pilot role abruptly switches from monitoring to active control, it is important that the vehicle have acceptable 
handling qualities. Extensive experience from aircraft development and operations has shown that pilot-monitored 
automatic control systems are safer when they are designed to operate in a manner consistent with human piloting 
techniques. This allows for improved monitoring as well as graceful reversion to manual control in the event of 
failures in the automatic control system. It is therefore desirable for spacecraft designers to assess early in the 
design cycle what the handling qualities will likely be, and to adjust their design if necessary to ensure that 
appropriate levels of handling qualities are available for both nominal and off-nominal operations. 

An effort to develop design guidelines for spacecraft handling qualities was initiated by NASA in 2007. A 
comprehensive set of guidelines should cover all classes of spacecraft and phases of flight; however, programmatic 
considerations made it necessary to focus initially on a few specific and relevant aspects. Lessons learned from 
aircraft and spacecraft handling qualities assessments over the past several decades were compiled. 16 Piloted 
simulation studies of lunar landing “ and Earth orbit docking were conducted. “ This paper covers an initial 
study on handling qualities for capsule entry, evaluating manual backup options after a failure of the automatic entry 
guidance/control system. 


II. Capsule Entry Operations 

For a spacecraft returning to Earth, the atmospheric entry phase of flight begins when it enters the atmosphere at 
approximately 400,000 ft altitude and 17,000 mph, following a deorbit burn. The subsequent trajectory of the 
spacecraft is governed by gravitational and aerodynamic forces - there are no propulsive forces. The aerodynamic 
forces acting on the spacecraft can be varied by reorienting the vehicle in the atmosphere using control surfaces, 
reaction control system (RCS) jets, or both. This in turn modifies the trajectory, enabling the auto/human pilot to 
steer the vehicle to the designated land/sea recovery site. 

Unlike winged spacecraft such as the Space Shuttle, capsule spacecraft generally do not have aerodynamic 
control surfaces. Typically, the aerodynamic forces acting on a capsule are varied by using RCS jets to reorient the 
vehicle in the atmosphere. During hypersonic flight, winged spacecraft have a lift-to-drag ratio, (L/D), on the order 
of 1; for capsule spacecraft the corresponding value is on the order of 0.3 so the capability for trajectory control is 
correspondingly less. Although capsules are generally axisymmetric in shape, they can generate some blunt-body 
lift at non-zero angles of attack (see Fig. 1), with (L/D) ~0.3 attained at roughly 20 deg angle of attack during 
hypersonic flight. However, the capsule must be trimmed to hold that angle of attack. This is generally 
accomplished (as in Apollo capsules) by precisely positioning the capsule center-of-mass (c.m.) away from the axis 
of symmetry so that the line of total aerodynamic force (i.e., resultant of lift and drag forces at the center-of-pressure 
(c.p.)) passes through the c.m. as shown in Fig. 1. The flight control system provides rate damping in the pitch and 
yaw axes to compensate for small disturbances. From a practical point of view, it is easier to offset the c.m. toward 
the equipment compartment rather than the crew compartment, and the resulting trim orientation of the capsule puts 
the crew compartment “upside down” as shown in Fig. 1. From the crew’s perspective, the spacecraft is moving 
“backwards” because the ablative heat shield, which is located behind the crew, must point forward for capsule 
entry. 

The mechanism for trajectory control during entry is to tilt the lift vector relative to the local vertical. This 
corresponds to a rotation of the vehicle about the air-relative velocity vector and is accomplished by applying a 
control moment about both the roll and pitch body axes. The rotation angle about the velocity vector is the bank 
angle (see Fig. 1); it determines the component of lift that opposes the gravitational force along the local vertical, 
and hence influences the acceleration along the local vertical given by: 


:• L cos u V 2 

h « — + g 

m R E + h 


( 1 ) 


where m is the vehicle mass, V is the vehicle speed, R E is the radius of the Earth, h is the altitude above Earth, and g 

is the acceleration due to Earth gravity. The rate of change of vehicle speed ( V « -Dim) depends primarily on the 
drag force, which is a function of the airspeed and the local atmospheric density which decreases with altitude. The 
magnitude of the “eyeballs in” g-load experienced by the crew depends primarily on the drag force. Consider an 
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entry trajectory with lift vector up {ji = 0). In this case, the lift force opposes the gravitational force to the maximum 
extent possible, and the resulting altitude profile is such that the capsule spends more time in the higher altitudes 
where the atmosphere is less dense. This results in a lower average drag force, which in turn results in the longest 
down-range distance and the lowest g-loads. Conversely, an entry with lift vector down (^ = 180 deg) results in the 
shortest down-range distance and the highest g-loads. Bank angle control allows adjustments to the down-range 
distance, but any bank angle other than zero or 180 deg will result in a side force on the capsule. Adjusting bank 
angle for down-range control, therefore, has the side effect of providing a cross-range distance. This cross-range 
distance builds up over time and can be adjusted with periodic bank reversals as necessary. The resulting targeting 
accuracy can be impressive; for example, Apollo 10 splashed down within a couple of miles of its recovery ship. 

During nominal entry operations, the desired bank angle is determined by the primary entry guidance system and 
then executed by the flight control system which fires the appropriate RCS jets to properly orient the vehicle; the 
crew’s function is to monitor the automated systems. The guidance system computes the desired bank angle 
(including bank reversals) using an algorithm that considers operational performance factors such as range -to-target, 
g-load profile, heating loads, and RCS propellant usage. If the primary guidance system fails, then guidance may 
revert to a backup system. For example, a “loads-managed” guidance system can compensate for limited/degraded 
navigation data by using a simpler algorithm whose primary focus is on keeping the g-loads profile within human 
endurance limits. Although it adjusts bank angle to compensate for disturbances, it may not perform bank reversals 
(to keep the guidance algorithm simple), which results in substantial cross-range errors. As a final backup capability 
(for example, if the loads-managed guidance system also fails) the spacecraft can be placed in a “ballistic mode” by 
simply spinning the capsule about its axis of symmetry at a rate of 10 - 15 deg/s with no capability to control the 
resulting trajectory. This mode results in very high g-loads and an uncontrolled (ballistic) trajectory. For example, 
in recent years, two separate Soyuz spacecraft had to make ballistic entries as a result of system malfunctions, 
landing hundreds of miles short of their recovery site. Also, ballistic entry was the nominal flight mode for Mercury 
capsules. Although all three types of entry trajectories described above can be flown automatically by the flight 
control system which fires the appropriate RCS jets to provide the desired vehicle attitude, the capability generally 
exists for the pilot to manually fly these types of entry trajectories. 


Local 

Vertical 


Drag 




Figure 1. Illustration of capsule entry dynamics 
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III. Experiment Design 

The principal objective of this experiment was to evaluate capsule handling qualities during atmospheric entry 
under manual control with different guidance options. The flight segment under study was from the entry interface 
(El) point to the drogue parachute deployment point. In this simulation, the target splashdown site was located 
roughly 200 miles west of San Diego, California. The El point was approximately 1,800 nmi south-west of the 
recovery site at 400,000 ft altitude; with an El speed of 16,950 mph, the time to drogue deployment was about 10 
min for the simulated trajectory. 

Entry trajectories are influenced by many environmental and vehicle parameters such as atmospheric conditions 
(pressure, temperature, winds), vehicle aerodynamics and mass properties, RCS jet properties, and navigation errors. 
The simulation featured two scenarios, characterized by nominal and highly dispersed values of many parameters 
including those mentioned above. 

Entry Performance 

Usage of the Cooper-Harper handling qualities rating scale 1 involves an assessment of pilot compensation to 
achieve a desired or adequate level of pilot- vehicle performance. The key operationally relevant performance 
parameters for entry trajectories are g-loads, propellant consumed, and range-to-target at drogue deployment. In this 
experiment, the g-loads constraint placed an upper limit on the time of exposure to sustained loads in excess of 3g, 
defined by a linear log-log relationship between the points (3g, 100 s) and (lOg, 10 s). There was an additional 
constraint on instantaneous g-load, with a limit of 8g for desired performance (lOg for adequate performance). 
Based on the results of autopilot and human-pilot test runs with different guidance configurations, the propellant 
usage limit was set to 90 lbm for desired performance (120 lbm for adequate performance). Limits on final range- 
to-target are discussed below. 

Piloting Tasks 

The simulation began in fast-time (i.e., non-real-time) mode with automatic control of the trajectory; it switched 
to real-time mode after about 3 minutes of simulation time (~15 seconds clock time) when the g-load built up to a 
value of approximately 0.17. Shortly thereafter, when the g-load reached a value of 0.2, the pilot took manual 
control; at this point the capsule was 940 nmi from the recovery site. The pilot continued flying the spacecraft until 
drogue deployment. 

The automatic control system provided rate damping in the pitch and yaw axes by firing the appropriate RCS 
jets. The pilot commanded a bank angle rate using a rotational hand controller (RHC). The maximum bank angle 
rate command was 20 deg/s at full displacement of the RHC, and at lower displacements it was proportionately less 
with some nonlinear shaping. The pilot command was converted by a flight control system mixer into the 
appropriate roll-rate and yaw-rate commands. These body rate commands were fed to the RCS jet firing logic which 
used one RCS jet to correct small bank-rate errors (< 2 deg/s) and two RCS jets to correct larger bank-rate errors. 
The firing logic provided the commanded bank angle rate within a dead-band of ±0.5 deg/s. By making small pulse- 
like RHC inputs, the pilot could adjust the bank angle to the desired value. 

There were two types of flying tasks: (1) bank angle guidance tracking for loads-managed entry, and, (2) flight 
parameter (vertical speed, Mach, energy) schedule following for targeted entry - there were two variants of this task. 

Loads-Managed Entry Task 

The pilot tracked closed-loop bank angle commands generated by the backup loads-managed guidance system 
whose primary objective was to keep the g-loads profile within human endurance limits. The piloting task was to 
correct the error between the commanded and actual bank angles, using an error needle on the cockpit display 
(details in next section). The guidance algorithm computed the commanded bank angle by searching a 5 deg grid, 
and the actual bank angle had some drift due to aerodynamic moments; hence the error signal typically featured 
gradual variations separated by occasional jumps on the order of 5 deg. The error signal in the segments between 
these jumps is referred to as the steady-state error. 

For this piloting task, the relevant performance parameter for the handling qualities assessment was steady-state 
error between commanded and actual bank angles. These error bounds were determined from fast-time Monte Carlo 
simulations with a simple (lag and bias) pilot model; this analysis showed that a bank angle error limit of 6 deg 
provided g-loads, final range-to-target and propellant usage that were within operationally acceptable bounds from 
the benchmark autopilot values. Accordingly, the bank angle tracking error limit was set at 6 deg for desired 
performance (10 deg for adequate performance). 

A key goal of the piloted simulation was to determine if the operationally relevant performance parameters 
(g-loads, final range-to-target and propellant usage) would have acceptable values if the manual piloting task was 
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simply to keep the steady- state bank angle errors within the specified limits. The performance goals for g-loads and 
propellant usage were as described in the previous sub-section. The benchmark autopilot runs resulted in final 
range-to-target values on the order of 60 nmi; hence the performance goal was a range error less than 70 nmi. The 
reason for the large range error is that were no bank reversals in this backup loads-managed guidance system’s 
commands, resulting in substantial cross-range error at drogue deployment. 

Targeted Entry Task 

The objective of this exercise was to evaluate the feasibility, in the absence of any closed-loop guidance, of 
manually steering the capsule close to the target site while meeting the operational constraints of g-load profiles and 
propellant budgets. In the absence of bank angle guidance commands for this task, the pilot used simple intuitive 
rules to determine the desired bank angle independently, based on an open-loop schedule of certain flight parameters 
specified at six range-to-target gates along the entry trajectory. These schedules were designed using a reference 
trajectory generated by the autopilot following commands from the primary entry guidance system for the nominal 
scenario. The flight parameters in the schedule were vertical speed, Mach, and total energy (potential plus kinetic). 
There were two types of schedule: a schedule of vertical speed only (called the h-dot schedule), and the full schedule 
of vertical speed, Mach, energy. These schedules were presented on the cockpit display (details in next section). 
The pilot monitored the evolution of the appropriate flight parameter(s) and adjusted the bank angle using intuitive 
rules of physics: raise the lift vector if sinking too fast and/or energy decreasing too rapidly; lower lift vector if 
sinking too slow and/or energy decreasing too slowly. Bank reversals were executed at four gates in the schedule to 
reduce cross-range error. A key requirement of this task is to get as close as possible to the recovery site target; 
hence it is called the targeted entry task. 

For this task, the performance parameters for the handling qualities assessment were g-loads, final range-to- 
target and propellant usage. The adequate performance limit for final range-to-target was set equal to the 
performance goal of the loads-managed entry task (i.e., 70 nmi), and the desired performance limit was set to 50% 
of the adequate limit (i.e., 35 nmi). The desired//adequate performance limits for g-loads and propellant usage were 
as described in the previous sub-section 

Experiment Matrix 

The principal objective of this experiment was to evaluate capsule handling qualities during atmospheric entry 
for a loads-managed entry task of following closed-loop bank angle guidance and a targeted entry task of following 
two types of open-loop schedule (h-dot and full). For each of these three tasks there were two scenarios 
corresponding to nominal and highly dispersed values of key parameters such as atmospheric conditions, capsule 
aerodynamics, c.m. location, RCS jet properties, and navigation errors. The resulting six configurations of the 
experiment matrix are depicted in Table 1. 


Flying Task -» 
Scenario Parameters j 

Loads 

Managed 

Entry 

Targeted Entry 

h only 

h. Mach. Energy 

Nominal 

✓ 

✓ 

✓ 

Dispersed 

✓ 

✓ 

✓ 


Table 1. Experiment matrix 


Evaluation Pilots 

Eight experienced test pilots served as evaluation pilots for this experiment; each had several thousand flight 
hours in a variety of fixed and rotary wing aircraft. Two were NASA pilots with decades of aircraft flight test and 
simulation experience. Six were retired NASA pilot astronauts from the Space Shuttle program with a combined 
space flight experience of 11 missions as Pilot and 14 missions as Commander. All the astronauts had piloted 
atmospheric entry in a winged spacecraft (Space Shuttle) and one astronaut had also experienced atmospheric entry 
in a capsule spacecraft (Soyuz). Each pilot was available for about 8 hours, and this time constraint was 
incorporated into the experiment design. 


5 

American Institute of Aeronautics and Astronautics 


Training Procedures 

Upon arrival, pilots received a detailed briefing on the experiment background and objectives, flying task, 
control system, test matrix, and data collection procedures. Including discussion time with the experimenters, this 
session lasted approximately one hour. This was followed by a training and familiarization session (about 1.5 hr) in 
the simulator cockpit, where pilots practiced the flying task for various representative configurations drawn from the 
test matrix until they felt comfortable that most of the learning curve was behind them. 

Data Collection Procedures 

Each pilot encountered the various experiment configurations in a different sequence and was not told whether 
the experiment scenario was nominal or dispersed. For each test configuration, the pilot flew two formal evaluation 
runs with an option for a third run if necessary (e.g., significant difference in performance across the first two runs). 
At the end of each run, relevant performance data were displayed to the pilot and experimenters. 

In handling qualities experiments, pilots are typically asked to make a composite assessment of the overall 
performance across all formal evaluation runs for a test configuration. This assessment takes into account not just 
the quantitative evaluation of the performance at the end point (e.g., drogue deployment) but also a qualitative 
evaluation of the manner in which the vehicle gets to the end point. This overall assessment of desired, adequate, or 
inadequate performance is utilized for traversing the decision tree in the Cooper-Harper chart. 1 Pilots use the 
Cooper-Harper scale to assign handling qualities ratings from 1 (best) to 10 (worst) based on their assessment of 
task performance and required compensation. It is an ordinal scale, which means for example that the difference 
between ratings of 1 and 2 is not the same as the difference between ratings of 3 and 4. Ratings of 1, 2, and 3 on the 
Cooper-Harper scale correspond to Level 1 handling qualities, which are a general requirement for normal 
operations of flight vehicles. Desired performance is necessary, but not sufficient, for Level 1 ratings. Ratings of 4, 
5, and 6 correspond to Level 2 which may be acceptable for some off-nominal conditions, and ratings of 7, 8, and 9 
correspond to Level 3 which is acceptable only for transition to a safe mode after a major failure or disturbance. 

After making a composite assessment of the overall performance across the formal evaluation runs for a test 
configuration, pilots assigned a handling qualities rating for that test configuration. The pilots also assigned ratings 
for each of the six components of the NASA Task Load Index. 27 These six components were: physical demand, 
mental demand, temporal demand, performance, effort, and frustration. The relative weighting of these six 
components for the flying task was determined by a pilot questionnaire at the end of the experiment. Workload 
ratings on the Bedford scale 28 were also assigned by the pilots. As appropriate, pilots provided qualitative 
comments about the test configuration they had just evaluated. All pilot commentary was recorded on electronic 
media while the experimenters separately noted key comments and observations. 

After all test configurations had been evaluated, there was a debrief session. The pilots filled out a one-page 
questionnaire designed to elicit high-level comments on cockpit displays, out-the-window displays, guidance cues, 
control response, and experiment design. This was followed by a discussion with the experimenters on the pros and 
cons of the various test configurations. 


IV. Simulation Environment 

The experiment was conducted on the Vertical Motion Simulator (VMS) at the NASA Ames Research Center. 
The VMS is a large-travel motion simulator 29 that has been used for numerous handling qualities evaluations. 30 Six- 
degree-of-freedom simulator motion was utilized for the experiment. A single pilot seat was installed in the center 
of the simulator cab, with a researcher/observer seat immediately aft of the pilot seat. A three-axis rotational hand 
controller (RHC) was installed on the right side of the pilot seat, and it was the only means of pilot input for this 
experiment. Although a three-axis translational hand controller (THC) was installed on the left side of the pilot seat, 
it was not used in this experiment, consistent with operating procedure for the entry piloting task. An illustration of 
the cockpit layout including the two control inceptors is shown in Fig. 2. 

A simulated view of the Earth and sky was projected on a set of three wide-angle collimated color displays with 
an image resolution of 1/2 pixel per arcmin. The window display had a large field of view: 48 deg vertical and 120 
deg horizontal. Window masking was not used in the simulator cockpit and therefore the entire field of view was 
available to the pilot. This is not representative of actual operations where the pilots have only a limited field of 
view through small windows. However, the entry piloting task in this experiment was primarily a head-down task, 
and the pilot’s attention was focused primarily on the cockpit instrumentation; the view outside the cockpit was used 
only for general situational awareness. 
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The cockpit console had three 6. 5 -inch color flat panel displays. The center panel was the primary flight display 
(see Fig. 3). The upper half of this panel displayed an Attitude Director Indicator (ADI) and tapes showing Mach 
number, angle of attack, altitude, and vertical speed (h-dot). Key parameters displayed in the lower half of the panel 
were g-load, percentage of RCS propellant remaining, and current range-to-target. The blue triangle on the ADI 
indicated the current bank angle; for example, if the indicator was at the top of the ADI then the lift vector was 
pointing up (away from the Earth’s surface) corresponding to zero bank angle. The yellow diamond and magenta 
needle were displayed only for the loads-managed entry task. The diamond indicated the bank angle guidance 
command, while the needle indicated the error between the commanded and actual bank angles. Pilots typically 
used the needle to correct bank angle errors under 10 deg. If the error was larger than 10 deg, the needle was caged 
and pilots used the triangle and diamond symbology to attenuate the error. The left panel displayed time histories of 
key flight parameters (g-load, bank angle, bank angle rate) at the end of the simulation run (see Fig. 4); it was blank 
during the run. The right panel served as a multifunction display. For the loads-managed entry task, this panel was 
blank during the simulation run. For the targeted entry task, it displayed data for range, vertical speed, Mach, and 
total energy, with the first data row showing current values and the data second row showing desired values at the 
next gate; current values of down-range and cross-range were shown at the bottom of the panel (see Fig. 4). At the 
end of the run, it displayed key parameters such as maximum g-load and position coordinates at drogue deployment; 
for the targeted entry task it also displayed (on a second page accessed by a toggle button) errors in vertical speed, 
Mach, and total energy at the schedule gates. 



Cockpit Displays 

Rotational Hand Controller 


"Out-the-Window" View 


Translational Hand Controller 


Figure 2. Simulator cockpit layout 


Dynamics and Control Model 

The simulation used an early version of the Orion vehicle model developed by NASA and Lockheed-Martin 
using the Advanced NASA Technology Architecture for Exploration Studies (ANTARES) software environment. 
The ANTARES simulation supports a wide variety of guidance, navigation and control engineering analyses. 24,25 
The VMS staff developed a software process to integrate the ANTARES model with the local simulation 
environment. A description of the Orion entry performance and mission design can be found in Ref. 26, while Ref. 
27 provides a description of the Orion entry simulation in ANTARES. The results reported in this paper do not 
constitute a formal evaluation of the actual Orion vehicle design; the goal of this study was to gain insight into entry 
handling qualities of the capsule class of spacecraft, using a preliminary Orion vehicle model as a means to that end. 


7 

American Institute of Aeronautics and Astronautics 



PFD Entry 


RET 000/00:06:12 


17.6 ■ 170 : 


17.4 


20QK 


16 5 


160 


17. 0M 


16.8 


15 5 


16.6 


Range to Target (nitn) 


RC S Prop % 

9 7-4 


UNITS English 
ERR SCALE 12 10 
RATE SCALE 25 


DAP: AUTO 


Ascent 


Figure 3. Primary flight display 
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Figure 4. Auxiliary flight displays 
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V. Results 


The data collection period in the VMS was 8-19 June 2009. The eight evaluation pilots provided Cooper- 
Harper ratings, NASA Task Load Index (TLX) ratings, Bedford workload ratings, and specific comments for each 
of the experiment configurations. They also provided their overall impressions of the manual entry tasks. For each 
run, time histories of numerous simulation variables were recorded along with key performance parameters at 
drogue deployment. A total of 94 formal evaluation runs were made during this experiment. This section presents 
qualitative and quantitative results obtained from these runs. 

Task Performance 

The performance parameters of interest for this experiment were: g-loads, range-to-target at drogue deployment, 
propellant usage, and bank angle tracking error (for loads-managed entry task only). The results showed that, for all 
formal evaluation runs, g-loads were within specified human endurance limits for both peak value and exposure 
time. 

Range error and propellant usage for all 94 formal evaluation runs, categorized by experiment configuration, are 
presented in Fig. 5; the circled data points indicate autopilot data. A detailed explanation of these results is provided 
below. 

Loads-Managed Entry Task 

For this task, the only performance parameter for the handling qualities assessment was steady-state bank angle 
error between commanded and actual values, where desired and adequate performance bounds corresponded to error 
magnitude less than 6 deg and 10 deg respectively. These error bounds were determined from fast- time Monte 
Carlo simulations with a simple (lag and bias) pilot model; this analysis showed that a bank angle error limit of 6 
deg provided range-to-target, g-loads and propellant usage that were within operationally acceptable bounds from 
the benchmark autopilot values. A key goal of the piloted simulation was to determine if the operationally relevant 
performance parameters (g-loads, final range-to-target and propellant usage) would yield acceptable values if the 
manual piloting task was simply to keep the steady-state bank angle errors within 6 deg. To limit propellant usage, 
pilots were advised not to correct errors under 2 deg. The experiment data showed that pilots were generally able to 
follow steady-state segments of the bank angle guidance command with an accuracy of 5 deg or better. 

Range errors were always close to their corresponding benchmark autopilot values of 59.7 nmi for the nominal 
case and 57.6 nmi for the dispersed case. The mean range errors for the piloted formal evaluation runs were 60.3 
nmi for the nominal case and 58.4 nmi for the dispersed case, but the difference between the corresponding data sets 
was not statistically significant (P-value > 0.05). The individual range errors were all less than 70 nmi. A large 
range error (mainly cross-range) was expected for the loads-managed entry task because there were no bank 
reversals. 

Propellant usage data showed a large spread relative to their corresponding benchmark autopilot values of 15 
lbm for the nominal case and 3 1 lbm for the dispersed case. The mean propellant usage values for the piloted formal 
evaluation runs were 41 lbm for the nominal case and 68 lbm for the dispersed case, and the propellant usage for 
piloted trajectories always exceeded the corresponding autopilot value. As expected, mean propellant usage for the 
dispersed case was substantially higher than that for the nominal case and the difference between the corresponding 
data sets was statistically significant (P-value < 0.05). Propellant usage was less than 90 lbm for 90% of the data 
points and always less than 120 lbm. 

The above results indicate that the operationally relevant performance parameters (g-loads, final range-to-target 
and propellant usage) yield acceptable values for loads-managed entry if the manual piloting task is simply to keep 
the steady-state bank angle error within ± 6 deg. For the overall data set of nominal and dispersed cases, the mean 
range errors were within 1% of the benchmark autopilot values, while the mean propellant usage was roughly 2.5 
times the benchmark autopilot values. This suggests that additional RCS propellant may need to be budgeted for 
this manual piloting contingency and that a looser guidance tracking limit may result in a better trade-off between 
range error and propellant usage. 

Targeted Entry Task 

Although this task had a steep learning curve, the pilots were able to develop a flying technique during the 
training period to meet the performance metrics. For this task, the performance parameters for the handling qualities 
assessment were range-to-target at drogue deployment, g-loads, and propellant usage. 

The experiment data showed that 94% of the data points had range errors within desired the performance bound 
of 35 nmi and all were well within the adequate performance bound of 70 nmi. For the h-dot schedule, the mean 
range errors were 23 nmi for the nominal case and 16 nmi for the dispersed case. For the full (h-dot, Mach, energy) 


9 

American Institute of Aeronautics and Astronautics 



schedule, the mean range errors were 16 nmi for the nominal case and 14 nmi for the dispersed case. Although the 
mean range error for the dispersed case was less than that for the nominal case for both types of schedules, the 
differences between the corresponding data sets were not statistically significant (P- value >0.05). Additionally, for 
both nominal and dispersed cases, the differences between range error data sets for h-dot schedule and full schedule 
were not statistically significant (P-value > 0.05). This indicates that the open-loop schedule, obtained from a 
primary-guidance-following autopilot run in the nominal scenario, is quite robust to large dispersions in key 
parameters such as atmospheric conditions, capsule aerodynamics, c.m. location, RCS jet properties, and navigation 
errors. 


Loads Managed 

Targeted Entry 

Targeted Entry 

Entry 

h_dot schedule 

Full schedule 

♦ Nominal 

• Nominal 

■ Nominal 

<» Dispersed 

© Dispersed 

B Dispersed 



Figure 5. Range error and propellant usage (circles indicate autopilot data) 
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Propellant usage was within the desired performance bound of 90 lbm for 77% of the data points, with only 2% 
of the data outside the adequate performance bound of 120 lbm. For the h-dot schedule, the mean propellant usage 
was 52 lbm for the nominal case and 95 lbm for the dispersed case. For the full (h-dot, Mach, energy) schedule, the 
mean propellant usage was 51 lbm for the nominal case and 81 lbm for the dispersed case. As expected, for both 
types of schedules, mean propellant usage for the dispersed case was substantially higher than that for the nominal 
case and the difference between the corresponding data sets was statistically significant (P-value < 0.05). For both 
nominal and dispersed cases, the differences between propellant usage data sets for h-dot schedule and full schedule 
were not statistically significant (P-value > 0.05). 

The above results indicate that the complex entry trajectory can be piloted using simple intuitive rules to follow 
an open-loop schedule, and this provides acceptable values of g-loads, final range-to-target and propellant usage. 
Since the difference between the range error data sets for nominal and dispersed cases was not statistically 
significant, it indicates that the open-loop schedule is robust to large dispersions in key parameters that influence 
entry trajectories. Overall, compared to loads-managed entry performance, targeted entry achieved a 71% lower 
range error at the expense of 27% more propellant usage primarily due to the use of bank reversals. 

Handling Qualities Ratings 

Handling qualities ratings for the six experiment configurations are presented in Fig. 6. In this bubble chart, the 
size of the bubble for a rating value indicates the number of pilots who assigned that rating, and a star symbol 
indicates the median rating. Each experiment configuration was rated by eight pilots, except for the dispersed loads- 
managed entry configuration which was rated by seven pilots. 
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Figure 6. Handling qualities ratings grouped by piloting task 
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Loads-Managed Entry Task 

The left portion of Fig. 6 shows data for nominal and dispersed cases for the loads-managed entry task. Most of 
the pilot ratings were Level 1, indicating satisfactory handling qualities. Pilot comments indicated that this was not 
a particularly challenging task. The primary, albeit not universal, complaint was about some high-frequency noise 
content in the guidance command which resulted in jittery movement of the error needle; this issue can be mitigated 
by filtering the guidance signal prior to display in the cockpit. 

For the loads-managed entry task, the only performance parameter for the handling qualities assessment was 
steady-state bank angle error between the commanded and actual values, where desired and adequate performance 
bounds corresponded to error magnitudes less than 6 deg and 10 deg respectively. Desired performance was almost 
always achieved - the data were typically showed an accuracy of 5 deg or better. The handling qualities rating of 6 
was assigned by a pilot who experienced mild oscillations while following the rather noisy guidance command too 
closely on one of his two formal evaluation runs for this configuration. This rating notwithstanding, the handling 
qualities ratings for the nominal configuration were slightly better than for the dispersed case. 

Targeted Entry Task 

The right portion of Fig. 6 shows data for nominal and dispersed cases for the two types of targeted entry tasks 
(h-dot schedule and full schedule). The pilot ratings were mostly Level 2, indicating that handling qualities warrant 
improvement. Suggestions for improvement included retaining data at the previous gate on the schedule display (in 
addition to data on current states and target values at the next gate), and displaying data on the primary display for 
acceleration along the local vertical (h-double-dot). Pilot comments indicated a preference for the option to use 
energy and/or Mach information in addition to h-dot information, and this is reflected by a slightly higher fraction of 
Level 1 ratings for the full schedule. 

For the targeted entry tasks, the performance parameters were range-to-target at drogue deployment, g-loads, and 
propellant usage. The performance data presented in the previous sub-section indicated that pilots achieved desired 
performance most of the time. The Level 2 ratings can hence be attributed primarily to the pilot compensation 
required for determining the bank angle to follow the specified open-loop schedule. However, the noteworthy result 
for this task is not the Level 2 rating - it is the finding that such a complex piloting task can be accurately performed 
by using a simple open-loop schedule that is robust to large dispersions in key parameters such as atmospheric 
conditions, capsule aerodynamics, c.m. location, RCS jet properties, and navigation errors. 
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Figure 7. Task Load Index component ratings 
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Workload Ratings 

NASA Task Load Index (TLX) component ratings for the six experiment configurations, averaged across all 
pilots, are presented in Fig. 7. The data show that the primary TLX components for this flying task were mental 
demand, temporal demand, and effort, while the secondary components were physical demand, performance, and 
frustration. The loads-managed entry task had the lowest workload components, and for targeted entry tasks the full 
schedule had slightly lower workload components than the h-dot schedule; this was true for both nominal and 
dispersed cases. For each of the three piloting tasks, there were only minor differences between the nominal and 
dispersed cases. 


VI. Conclusions 

An evaluation of handling qualities for atmospheric capsule entry was conducted by eight pilots, including six 
astronauts, flying the NASA Ames Vertical Motion Simulator. The piloting tasks in this experiment were a bank 
angle guidance command tracking task for loads-managed entry, and two types of flight parameter (vertical speed, 
Mach, energy) schedule following tasks for targeted entry. Each piloting task included two scenarios featuring 
nominal values and high dispersions for key parameters such as atmospheric conditions, capsule aerodynamics, c.m. 
location, RCS jet properties, and navigation errors. Performance parameters of operational interest were range-to- 
target at drogue deployment, g-loads, and RCS propellant usage. In all cases, the g-loads were found to be within 
human endurance limits. 

For the loads-managed entry task, the pilot followed closed-loop bank angle guidance commands generated by 
the backup loads-managed guidance system. The piloting task was to correct the error between the actual and 
commanded bank angles based on visual cues presented on the cockpit displays. Pilots were able to track steady- 
state segments of the guidance commands with an accuracy of 5 deg or better. With this level of tracking accuracy, 
the mean range-to-target values at drogue deployment were within 1 % of the benchmark autopilot values, while the 
mean propellant usage was roughly 2.5 times the benchmark autopilot values. These results indicate that additional 
RCS propellant may need to be budgeted for this manual piloting contingency, and that a looser guidance tracking 
limit may result in a better trade-off between range error and propellant usage. Overall, the handling qualities 
ratings for task were Level 1 . Pilot comments indicated that this was not a particularly challenging task. 

For the targeted entry task, the pilot used simple intuitive rules to determine the desired bank angle, based on an 
open-loop schedule of certain flight parameters specified at several range-to-target gates along the entry trajectory. 
The flight parameters were vertical speed, Mach, and total energy. There were two types of schedule: a schedule 
for vertical speed only (h-dot schedule), and the full schedule of vertical speed, Mach, and energy. The piloting task 
was to determine bank angle adjustments by using schedule data presented on the cockpit displays and applying 
intuitive rules, e.g., raise lift vector if sinking too fast. Although there was a steep learning curve for this task, the 
pilots were able to develop a flying technique during the training period and get good scores on the performance 
metrics. Since this task included bank reversals, the range error was about 71% lower than that for the loads- 
managed entry task at the expense of about 27% higher propellant usage. Overall, the handling qualities ratings for 
this task were Level 2. Since desired performance was achieved most of the time, the Level 2 ratings can be 
attributed primarily to the pilot compensation required for determining the bank angle to follow the specified open- 
loop schedule. Pilot comments for improvement include displaying schedule information for the previous gate and 
displaying data for acceleration along the local vertical (h-double-dot). The ratings were slightly better for the full 
schedule and pilot comments indicated a preference for the option to use energy and/or Mach information in 
addition to h-dot information. The noteworthy result for the targeted entry task is not the Level 2 ratings - it is the 
finding that such a complex piloting task can be performed accurately using a simple open-loop schedule that is 
robust to large dispersions. 
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